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Summary 

Instrumental variable (IV) methods are widely used to adjust for the bias in estimating treat¬ 
ment effeets eaused by unmeasured eonfounders in observational studies. In this manuseript, 
we provide empirieal and theoretical evidence that the IV methods may result in biased treat¬ 
ment effects if applied on a data set in which subjects are preselected based on their received 
treatments. We frame this as a selection bias problem and propose a procedure that identifies 
the treatment effect of interest as a function of a vector of sensitivity parameters. We also list 
assumptions under which analyzing the preselected data does not lead to a biased treatment ef¬ 
fect estimate. The performance of the proposed method is examined using simulation studies. 
We applied our method on The Health Improvement Network (THIN) database to estimate the 
comparative effect of metformin and sulfonylureas on weight gain among diabetic patients. 
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1. Introduction 

Confounder adjustment is critical in the estimation of treatment effect in observational studies. 
In practice, however, there is no guarantee that all the eonfounders (i.e., baseline variables that 
affect both the treatment and outcome) have been measured. These unmeasured eonfounders can 
bias the treatment effect estimation. 

Instrumental variable (IV) methods are designed to adjust for the bias caused by unmeasured 
eonfounders in estimating a difference measure of effect such as the risk difference. IVs are pre¬ 
dictors of treatment allocation that are independent of unmeasured eonfounders and do not have 
a direct effect on the outcome (i.e., only affect the outcome through affecting the treatment). 
Intuitively, the IV method seeks to extract variation in treatment that is free of unmeasured con- 
founders and uses this variation to estimate the treatment effect. For more information on IV 
methods, see Angrist et al. (1996), Newhouse and McClellan (1998), Greenland (2000), Hernan 
and Robins (2006), Cheng et al. (2009a), Baiocchi et al. (2014) and Imbens (2014). This paper 
is motivated by provider preference IV (PPIV) which is commonly used as an IV in health stud¬ 
ies. PP IV utilizes the natural variation in medical practices to construct an IV. For example, in 
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studying the effect of cyclooxygenase-2 (COX-2) inhibitors vs. nonselective nonsteroidal anti¬ 
inflammatory drugs (nonselective NSAIDs) on gastrointestinal complications Brookhart et al. 
(2006) used whether a patient’s physician last prescription for a patient prescribed a nonsteroidal 
anti-inflammatory drug (NS AID) was for a Cox-2 inhibitor or a nonselective NSAID as an IV for 
whether the patient was prescribed a Cox-2 inhibitor or a nonselective NSAID. Other examples 
of PP IV can be found in Brooks et al. (2003) and Brookhart and Schneeweiss (2007). 

For a binary IV and binary treatment, Angrist et al. (1996) proposed a nonparametric estimator 
which identifies the treatment effect among subjects who would take the treatment if encouraged 
to do so by the IV and not take the treatment if not encouraged to do so by the IV (compliers). 
This estimator is equivalent to a 2 stage least square (2SLS) estimator where at the first stage the 
treatment is regressed on IV and at the second stage the outcome is regressed on fitted values of 
the first stage regression (i.e., predicted value of the treatment variable given the IV). The coef¬ 
ficient of the independent variable of the second stage regression is equivalent to the estimator 
proposed by Angrist et al. (1996). The 2SLS estimator is also called the Wald estimator after 
Wald (1940). More efficient versions of the 2SLS estimator has been developed by Imbens and 
Rubin (1997b), Imbens and Rubin (1997a) and Cheng et al. (2009b). 

Although treatment effect estimation using IV approaches for binary treatments is straight¬ 
forward, it is challenging when there are more than two treatment arms. In fact, Cheng and 
Small (2006) shows that the treatment effects are not point identifiable and proposes bounds on 
treatment effects using method of moments. Long et al. (2010) studies the same problem and 
proposes a Bayesian approach for finding credibility intervals for the identification region. 

Sometimes investigators are interested in comparing the effect of two treatments while the data 
consist of more than two treatment options for patients. In this situation a common approach is 
to select patients who are assigned to the treatments of interest and perform the standard IV 
approach for binary treatment and IV to estimate the treatment effect (Hadley et ah, 2003; Basu 
et ah, 2007; Hadley et ah, 2010; Sub et ah, 2012; Chen et ah, 2014). 

The objective of this manuscript is to provide empirical and theoretical evidence that, in gen¬ 
eral, excluding patients based on their assigned treatment can result in selection bias. This prob¬ 
lem has also been discussed in an independent work by Swanson (2014a), Swanson (2014b), 
and Swanson et al. (2015). In the current manuscript, we formalize the mechanism of this selec¬ 
tion bias within a principal stratification framework and provide a procedure that identifies the 
treatment effect of interest as a function of a vector of sensitivity parameters. Specifically, the 
sensitivity parameters are used to estimate the probability of being assigned to one of the treat¬ 
ments of interest as a function of the observed outcome and the IV value (Gilbert et ah, 2003). 
We also list assumptions under which selecting patients based on their received treatment does 
not lead to a biased treatment effect estimate. 

The manuscript is organized as follows. Section 2 provides the setup and defines the causal 
estimand. It also lists the required assumptions for our procedure. In Section 3, we discuss the 
selection bias issue and present the proposed sensitivity analysis method. We report on the results 
of a simulation study in Section 4, where we examine the performance of our proposed method. 
In Section 5, we study the comparative effect of metformin and sulfonylureas on weight gain 
using The Health Improvement Network (THIN) database. 


2. Settings and Notations 

For simplicity we focus on observational studies with three treatment options A, B, and C. We 
assume that we are only interested in comparing the effect of treatment A with B. We use small 
letters to refer to the possible values of the corresponding capital letter random variable. Let Z be 
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a binary instrumental variable where Z G {A,B} sueh as provider preferenee. Z = A means that 
the IV predisposes the patient to reeeive treatment level A among the options A and B, and Z = B 
means that the IV predisposes the patient to reeeive treatment level B among the options A and 
B; e.g., in Brookhart et al. (2006) study, if A denotes Cox-2 inhibitor and B denotes nonseleetive 
NSAID, Z = A if the patient’s physieian last preseription to a patient preseribed an NSAID was 
a Cox-2 inhibotor and Z = B if it was a nonseleetive NSAID. Let D{z) be the potential treatment 
assigned given Z = z, where D{z) G {A,B,C}, i.e., D(A) is the treatment that would be reeeived 
if the IV was A. The potential outeomes are for Z = z and D = d, i.e., is the outeome 
that would be observed if the IV was A and the treatment level was B. The observed treatment 
and outeome are D = and Y = T^’^, respeetively. 

Frangakis and Rubin (2002) developed a basie prineipal strata (PS) framework whieh stratifies 
subjeets based on their potential assigned treatments. In our setting, we ean elassify subjeets into 
the following 9 prineipal strata: SI) always B taker, S2) always A taker, S3) always C taker, S4) 
subjeets who would eomply with their value of instrumental variable, i.e., D(A) = A, D{B) = B, 
S5) subjeets who would take B if Z = B and would take C if Z = A, S6) subjeets who would take 
C if Z = B and would take A if Z = A, S7) subjeets who would take A if Z = B and would take 
B if Z = A, S8) subjeets who would take B if Z = A and would take C if Z = B, and S9) subjeets 
who would take A if Z = B and would take C if Z = A. We denote the proportions in the basie 
prineipal strata as 71^'. 

We assume that the IV satisfies fhe following sfandard assumptions (Angrisf el al., 1996): 1) 
slable unil Irealmenl value assumpfion, 2) The IV is independenl of unmeasured eonfounders, 
polenlial oufeomes and pofenfial frealmenf reeeived, 3) fhe IV affeels fhe Irealmenl bul has no 
direel effeel on Ihe ouleome. This assumption is known lo as Ihe exelusion reslrielion (ER) 
assumption. Under Ihe exelusion reslrielion, we lei Y‘^ be Ihe potential outeome if Irealmenl d is 
reeeived whieh equal 7 ^= 0 ,rf _ jY independenl of Ihe prelrealmenl eovariales, 

potential outeome and potential Irealmenls. Furlhermore, we assume Iwo lypes of monolonieily 
assumptions: 5) monolonieily I: Ihere is no defiers (i.e., p{D{A) / A,D{B) / B) = 0) and 6) 
monolonieily II: Ihere is no one who would lake B if Z = A bul lake C if Z = B and no one who 
would lake C if Z = A bul lake A if Z = B. Note lhal monolonieily assumptions 5 & 6 reduee Ihe 
number of prineipal slrala lo 6. The parameter of inleresl is defined as 

e = E[T^-T^|B5 = 54]. 


3. IV Selection Bias 

We show in this manuseript that IV analysis methods may not result in an unbiased treatment 
effeet if applied on a data set in whieh units are preseleeted based on their reeeived treatments. 
A standard IV analysis of patients on the subset of patients for whom D = A or B is essentially 
a eomparison of the Z = A vs. Z = B patients on this subset. If Z = A has a different effeet on 
whether D = C than Z = B, then there will be seleetion bias. For example, patients who are in a 
more severe stages of the disease may be treated with treatment D = C and therefore have less 
ehanee to be observed in the sample; if doetors who prefer drug A (Z = A) are more likely to 
use treatment C with severe patients than doetors who prefer drug B (Z = B), then there will be 
seleetion bias. Another seenario where sueh seleetion bias might arise is that treatment C is more 
likely to be offered if some eomplieations are present and doetors who prefer drug A are more 
likely to treat a patient with eomplieations with drug C than doetors who prefer drug B. 

The ehallenge is that d is not identifiable using the observed data when there is seleetion bias. 
This is due to the faet that the observed outeomes of T|Z,D is a mixture of potential outeomes 
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from principal strata. Let /a (y) be the density of outcome for a patient who received D = A when 
assigned to Z = A (i.e., /^(y) = f{y\D = A,Z = A). Then 

fA{y) = (y) + (y), 


where = ^s 2 ^^s 4 _^_j,;S 6 and (y) is the density function of the outcome for subjects in principal 
stratum S. with D = A. The density f^{y) and f^{y) can be written as 


. 52 . . ^ p(D(fi) =A|D(A) =A,T(A) =y)/4(y) 
p{D{B)=A\D{A)=A) 

.56. . ^ P{D{B)=C\D{A) =A,T(A) =y)/A(y) 
p{D{B) = C\D{A) = A) 

Thus, 


fl^y) 


[l-^p{D{B)=C\D{A) =A,Y(A) =y)-p{D{B) =A\D{A) =A,Y{A) =y)]fA{y) 

f/ 


( 1 ) 


Similarly, let /^(y) be the density of outcome for a patient who received D = B when assigned 
to Z = B. Then 


fsiy) = i^Vl^y) + T^Vf (y) + T^Vf (y), 


where and (y) is the density function of the outcome for subjects in principal 

stratum S. with D = B. Also, 


.51. . ^ P(T>(A) =B|D(B) =B,T(A) =y)/4(y) 
p{D{A) = B\D{B) = B) 

.55. . ^ p{D{A)=C\D{B)=B,Y{B) =y)fB{y) 
p{D{A) =C\D{B) =B) 

Thus, 


^54 ^ [1 ^ p{D{A) = C\D(B) = B, Y{B) =y)- p(D{A) = B\D(B) = B, Y{B) = y^jy) 

^ Yb 


( 2 ) 


The mixture proportions y^' and y^' are unknown and need to be estimated using the observed 
data. However, the strata proportions are not identifiable using the proportions in the observ¬ 
able {D,Z) strata, because 

pl^D = A\Z = A) = 71^^ + + 71^^ 

p{D = A\Z = B) = 71^^ 

p(D = B|Z=A) = 71^^ 

p{D = B\Z = B)= 71^^ + + 71^^ 

p{D = C\Z = B) = 7i:^^ + 7i:^^ 

p{D = C\Z=A) = 7i:^^ + 7i:^^, ( 3 ) 

does not have a unique solution. Also, the density functions f^{y) and fg^{y) are not identifi¬ 
able using fhe observed dafa because p{D{B) = A|D(A) = A,T(A) = y), p{D{B) = C|D(A) = 
A,Y{A) = y), p{D{A) = B\D{B) = B,Y{B) = y) and p{D{A) = C\D{B) =B,Y{B) =y) are un¬ 
known. So we assume logif models for fhese probabilifies and perform fhe sensifivify analysis 
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Fig. 1: Causal diagram. Conditioning on subjects with /? = 1 leads to a bias treatment effect esti¬ 
mate in the left figure. Assumptions A. 1 and A.2 hold for the center and right figure, respecfively. 


based on fhese models; 


gabA+aur 

p{D{B)=B\D{A) =A,Y{A) = y) = WA(y; Ooa, «u) = (4) 

„a(,B+a\By 

p{D{A) =A\D{B)=B,Y{B) = y) = Oob, ais) = (5) 

Note fhaf for any fixed {aiA,OL\B), {(X 0 A 7 CC 0 B) can be determined since JfA^{y)dy = 1 and 
/ fB^{y)dy = 1. Therefore, fhe paramefer of inferesf d can be esfimafed using fhe four dimen¬ 
sional sensifivify paramefer au, Otis). 

Remark. If if is plausible fo assume fhaf af teas! one of fhe principal sfrafa or is 

empfy, we can reduce fhe dimension of fhe sensifivify parameters fo fwo by estimating and 
y^^ using fhe observed dafa. 


3T. No Selection Bias 

Lef be a binary variable which is 1 if a pafienf is assigned fo one of fhe freafmenfs of inferesf 
and 0 ofherwise. Under eifher of fhe following fwo conditions, fhe freafmenf effecl is idenlifiable 
when we have access fo fhe entire dafa (i.e., dafa includes pafienfs who have received freafmenf 
C as well as A and B): 

A. 1 p{R = 0|Z, U,%) = p{R = 0| U, X) where X and U are measured and unmeasured confounders, 

respecfively. 

A.2 p{R = 0\Z,U,X)=p{R = 0\Z,X). 

Assumption A.l implies fhaf fhe IV is unrelafed fo fhe pafienf’s selection. In ofher words, a 
pafienf would have fhe same chance of being selecfed for bofh values of fhe IV. A.2 means fhaf 
fhe selection probabilify is independenf of fhe unmeasured confounders given fhe insfrumenfal 
variable and measured confounders. Figure 1 depicfs fhe association befween differenl variables. 
The lefl plof presenfs a scenario in which analyzing fhe preselected dafa will cause bias because 
condifioning on R will open fhe pafh Z ^ R^U -^Y which violafes one of fhe IV assumptions. 
This pafh is closed under eifher of assumpfions A.l or A.2 as shown in fhe cenfer and righf plofs, 
respecfively. 

Under eifher of assumpfions A.l or A.2, fhe frealmenf effecl can be idenlified. Specifically, 
under assumpfion A.l, fhe principal sfrafa 55 and 56 does nol exisl. Thus 

E[T|Z = A,D = A] = /2E[T^|P5 = 52]-h/4E[T^|FS = S4] 

E[Y\Z = B,D = B\ = /^E[T^|FS = Sl]-h/'‘E[T^|FS = S4] 

E[T|Z = A,D = B] = /^E[T^|FS = S1] 

E[T|Z = B,D = A] = /^E[T"^|FS = S2]. 
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These equations ean be used to identify 6 = E[T^ — Y^\PS = 54]. Also, under assumption A.2, 
6 ean be estimated using the following three-step proeedure: 

Stepl. Using the entire data, estimate dJ = p{R = 1 |X,Z) by postulating a logit model on S. 

Step2. Inelude patients who have been assigned to either treatments A or B. 

Step3. Estimate the treatment effeet 6 using 2SLS IV approaeh. At eaeh stage fit a weighted least 
square where weights are GT. 

Fitting weighted least squares aeeounts for the seleetion proeedure and henee avoids the bias 
by ereating a pseudo-population in whieh there is no assoeiation between (X,Z) and R. 


3 • 2. Inference Procedure 

Let Ya = (Tti, Ya 2 Yah,^ ) and Yb = (Ybi , Yb 2 , • • •, Ybhb ) denote the outeome of patients whose 
reeeived treatment matehes their IV value for Z = A and Z = B, respeetively. We assume that 
samples are independent within and between eaeh IV group and identieally distributed within 
eaeh group. 

For any fixed , Wia, otie), (Sqa, Sob) is defined as a solufion fo 



gOoA + CClAy 
\ q-gOtoA+aur 
gOOB+CClB)' 

\ _|_ gOtOB + dlBy 


dh{y) = 7l‘' 
dPsiy) = yp 


where pAiy) and FB{y) are empirieal disfribufions ealeulafed from fhe observed samples Ya and 
Yb- These equafions ean be solved using a one dimensional grid seareh on real line. 

We esfimafe fhe freafmenf effeef d by 


d = 


1 ma ^ 

-54 — XI A , aiA) 

M j=i 



riB 

X ysjWBiyBj', «0B, o:ib) 


where wa(.) and wb(.) are defined in (4) and (5). By boofsfrap sampling from Ya and Yb, we 
ean esfimafe fhe sfandard error of 6 as fhe sfandard error of fhe esfimafed freafmenf effeef from 
eaeh boofsfrapped sample. 


4. Simulation Studies 

Through simulafions we examine fhe performanee of our proposed sensifivify analysis assum¬ 
ing known and unknown We assume fwo seenarios wifh fhe following prineipal sfrafa 

proportions: Seenario 1: 

• Seenario 1: = 0.1, = 0.3 and 71 ^^ = 0.3 whieh gives us (yf^, yf^) = 

/OA 03\ 

lo.v’ 0 . 5 T 

• Seenario 2: 71^^ = 71^^ = 71^^ = 0.1, 7l^‘^ = 0.3, 71^^ = 0.4 and 71^^ = 0.0 whieh gives us 

(yS4 ^ 4 ') _ (02 02 ') 

\IA ’ IB J V0.3 ’ 0.7 Y 

The frue amounf of seleefion bias is defermined by fhe paramefers a\A and a^B in (4) and (5), 
respeetively. The oufeome of individuals wifh D = A and Z = A are generafed from a normal 
disfribufion wifh mean 2.5 and sfandard error 2 (i.e., /A(y) = V(2.5,2)). Also, fhe oufeome of 
individuals wifh D = B and Z = B are generafed from a normal disfribufion wifh mean jj. and 
sfandard error 2 (i.e., /B(y) = V(/l,2)). For eaeh sef of (otiA, 0C\b), the parameter jj. is determined 
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Fig. 2: Simulation study: Sensitivity analysis of 0 = 0.80 when and are known. The 
true sensitivity parameter values are (aiAjOtis) = (1,2). The darker area in left panel indicates 
the smaller bias and the darker area in the right panel shows the region in which the bias is not 
significant at 5%. The first and second rows represent Scenario 1 and 2, respectively. 


such that the true treatment effect among compilers is 6 = 0.00, 0.50 and 0.80. We generate 
samples using rejection sampling from densities f^^{y) and fg^{y) according to (1) and (2), 
respectively. 

Table 1 presents the statistical power of detecting the true treatment effect 0 = 0.0, 0.50, 
and 0.80 based on 500 datasets of sizes n=500 with (aiAjOtifi) = (1,2), (1,1) and (0,0). The 
nominal type-I error rate is 5%. For both scenarios when the true amount of selection bias 
is presumed, the empirical and nominal sizes are close. In general, the empirical sizes are in¬ 
flated when the selection bias parameters are not set correctly. For some of the combinations of 
the true and presumed sensitivity parameter values the power and type-I error rate are much 
higher than the nominal one which is caused by the overestimation of the treatment effect. 
Also, in scenario 2, when there is no selection bias (i.e., true {a\A-,(X\B) = (0,0)) and we pre¬ 
sumed {a\A-,(X\B) = (1,1) the power is very low because of the underestimation of the treat¬ 
ment effect. Table 2 shows the bias (0.80 — 0), standard error and mean squared error of the 
treatment effect estimation for different combinations of the true and presumed sensitivity pa¬ 
rameters when the true treatment effect 0 = 0.80. This table highlights the over- and underes¬ 
timation phenomenon discussed earlier. Specifically, when fhe frue sensifivify paramefers are 
(aiA,o;iB) = (1,2), presuming {a\A-,Oi\B) = (0,2) will resulf in underesfimafing fhe counlerfac- 
fual mean oufcome E[FLIPS' = 54]. This is because when we sef fhe presumed a\A less fhan fhe 
frue value, fhe larger oufcomes are less likely fo be a compiler among pafienfs wifh D = Z = A. 
In ofher words if will shifl fhe disfribufion of fhis compilers fo lefl. Similarly when fhe presumed 
a\A more fhan fhe frue value, E[FLIPS' = 54] will be overesfimafed (See rows 3 and 4 in Table 
2 ). 

Figure 2 shows how fhe frealmenf effecl esfimafe changes by fhe sensifivify paramefers when 
fhe frue (ttiAjOtig) = (1,2) and 0 = 0.80. The darker area in lefl panel of Figure 2 indicates 
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Fig. 3: Simulation study (Scenario 1): Sensitivity analysis of 0 = 0.80. Sensitivity parameters 
are {y^,a\A-,Oi\B)- The darker area in left panel indicates the smaller bias and the darker 
area in the right panel shows the region in which the bias is not significant at 5%. The true vector 
of sensitivity parameters are ttu, Otis) = (|,|,1,2). 


the smaller bias and the darker area in the right panel shows the region in which the bias is not 
significant at 5% (i.e., the confidence interval for (0.80 — Q) contains zero). See also Table 2. 

4-1. Unknown (yf^,yg‘^) 

In this case our sensitivity parameter space is (y^^, , ttiA, Otis). Figures 3 and 4 present the 

bias in the treatment effect estimation for different values of the presumed sensitivity parameters 
based on scenarios 1 and 2, respectively. In these figures the true (y^^, y^^, Wia, Otis) = (|, |, 1,2) 
and (y^^,ttiA,Otis) = (5,|,I,2) in scenarios I and 2, respectively. Also the true treatment 
effect is 6 = 0.80. The darker area in the left panel shows the region in the sensitivity parameter 
space in which the bias is smaller and right panel shows the area in which the bias is not signifi¬ 
cant based on 500 samples (i.e., the confidence interval for (0.80 — 0) contains zero). The shape 
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Fig. 4: Simulation study (Scenario 2): Sensitivity analysis of 0 = 0.80. Sensitivity parameters 
are ,a\A,Oi\B)- The darker area in left panel indicates the smaller bias and the darker 

area in the right panel shows the region in which the bias is not significant at 5%. The true vector 
of sensitivity parameters are = (|,^,1,2). 


of the area which leads to an unbiased estimate changes by the value of the presumed 
and, in general, when the presumed values of {yf^,yB^) are far from the true values, the unbiased 
area is small which means it would be less likely to find an unbiased treatment effect estimate 
using sensitivity analysis (see also Table 4 in Appendix A). We have tabulated the power and 
type-I error rate for different values of the presumed sensitivity parameters in Table 3. This Ta¬ 
ble summarizes the result for three different values of the true treatment effect d = 0.00,0.50, 
and 0.80. The nominal type-I error rate is 5%. 






10 


Table 1: Simulation study: Power analysis of 0 = 0.00,0.50 and 0.80 when and are 
known. 


Scenario 1 Scenario 2 


True (aiA,aiB) 

Presumed {aiA,(XiB) 

0.00 

0.50 

0.80 

0.00 

0.50 

0.80 

( 1 , 2 ) 

( 1 , 2 ) 

0.04 

0.60 

0.91 

0.04 

0.52 

0.86 

(1,2) 

(1,1) 

0.12 

0.81 

0.98 

0.22 

0.88 

0.97 

(1,2) 

(0,0) 

0.16 

0.36 

0.78 

0.99 

1.00 

1.00 

( 1 , 1 ) 

( 1 , 1 ) 

0.05 

0.58 

0.91 

0.05 

0.48 

0.85 

(1,1) 

(2,2) 

0.08 

0.71 

0.95 

0.10 

0.24 

0.61 

(1,1) 

(0,0) 

0.35 

0.05 

0.40 

0.90 

1.00 

1.00 

( 0 , 0 ) 

( 0 , 0 ) 

0.04 

0.61 

0.92 

0.05 

0.60 

0.94 

(0,0) 

(1,0) 

1.00 

1.00 

1.00 

0.80 

1.00 

1.00 

(0,0) 

(1,1) 

0.34 

0.96 

1.00 

0.81 

0.00 

0.03 


Table 2: Simulation study: Sensitivity analysis of 0 = 0.80 when and yp are known. Bias= 
0.80-0. 


Scenario 1 Scenario 2 


True (aiA,aiB) 

Presumed (aiAjOtis) 

Bias 

S.D. 

MSB 

Bias 

S.D. 

MSB 

(1,2) 

(1,2) 

-0.01 

0.28 

0.08 

0.01 

0.30 

0.09 

(1,2) 

(1,1) 

-0.22 

0.29 

0.14 

-0.38 

0.32 

0.25 

(1,2) 

(0,2) 

1.38 

0.25 

1.97 

0.81 

0.29 

0.74 

(1,2) 

(1,0) 

-1.18 

0.25 

1.43 

-2.17 

0.28 

4.82 

(1,2) 

(0,0) 

0.21 

0.23 

0.10 

-1.34 

0.25 

1.86 

(1,1) 

(1,1) 

0.00 

0.28 

0.08 

-0.01 

0.30 

0.10 

(1,1) 

(1,2) 

0.19 

0.28 

0.11 

0.39 

0.31 

0.26 

(1,1) 

(0,2) 

1.58 

0.27 

2.58 

1.17 

0.30 

1.49 

(1,1) 

(1,0) 

-0.98 

0.26 

1.02 

-1.77 

0.28 

3.20 

(1,1) 

(0,0) 

0.41 

0.23 

0.23 

-0.95 

0.27 

0.98 


5. Application to Real Data 

The Health Improvement Network (THIN) is a large database, which contains the electronic 
medical record of more than 11 million patients. The data contains longitudinal measurements 
of diagnostic and prescription data and baseline characteristics collected from over 500 general 
practices in the UK. We are interested in assessing the effect of metformin and sulfonylureas 
on body mass index (BMI, calculated as mass in kilograms divided by the square of height in 
meters) among diabetic patients. Our focus is on patients who are taking these treatments as an 
initial treatment and the BMI is measured two years after treatment initiation. In our analysis we 
coded metformin and sulfonylureas as A and B, respectively. 

In the modern era, metformin is universally acknowledged as the appropriate first-line medica¬ 
tion for diabetes type 2 except where contraindicated (Nathan et al., 2009). While this dominance 
was being established in the late 1990s and early 2000s, sulfonylureas were the competing first 
line agent. During that time the prevalence of metformin use in clinical practice rose very quickly, 
at the expense of sulfonylureas. 
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Table 3: Simulation study: Power analysis of 0 = 0.00,0.50 and 0.80. The true veetor of sen¬ 
sitivity parameters in seenario 1 and 2 are = (|,|,1,2), and (|,|,1,2), re- 

speetively. 


Presumed Seenario 1 Seenario 2 



0.00 

0.50 

0.80 

0.00 

0.50 

0.80 

(0.3,0.6,1,2) 

0.38 

0.90 

0.98 

1.00 

1.00 

1.00 

(0.3,0.6,0,2) 

1.00 

0.00 

0.00 

0.07 

0.73 

0.96 

(0.3,0.6,1,0) 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

(0.3,0.6,0,0) 

0.12 

0.35 

0.80 

1.00 

1.00 

1.00 

(0.5,0.5,1,2) 

0.40 

0.08 

0.38 

0.93 

1.00 

1.00 

(0.5,0.5,0,2) 

1.00 

0.00 

0.00 

0.04 

0.40 

0.75 

(0.5,0.5,1,0) 

0.96 

1.00 

1.00 

1.00 

1.00 

1.00 

(0.5,0.5,0,0) 

0.10 

0.29 

0.76 

1.00 

1.00 

1.00 

(0.2,0.8,1,2) 

0.98 

1.00 

1.00 

1.00 

1.00 

1.00 

(0.2,0.8,0,2) 

0.94 

0.00 

0.05 

0.61 

1.00 

1.00 

(0.2,0.8,1,0) 

1.00 

1.00 

1.00 

1.00 

1.00 

1.00 

(0.2,0.8,0,0) 

0.06 

0.24 

0.65 

1.00 

1.00 

1.00 

(0.7,0.2,1,2) 

1.00 

0.00 

0.00 

0.30 

0.08 

0.32 

(0.7,0.2,0,2) 

1.00 

0.00 

0.00 

0.98 

0.00 

0.00 

(0.7,0.2,1,0) 

0.65 

0.99 

1.00 

1.00 

1.00 

1.00 

(0.7,0.2,0,0) 

0.12 

0.39 

0.87 

1.00 

1.00 

1.00 


The dynamie aspeet of the preferenee justifies the use of provider preferenee as an IV. We 
define fhe preferenee as a time-varying quanfify sueh fhaf for eaeh praefiee ID, fhe IV is defined 
as an average of mefformin use during eaeh fwo year timeframe from 1998 fo 2012. We assign 
Z = A if fhe average is more fhan 50% and B ofherwise. Thus, a parfieular praefiee ID may have 
Z = A for one period of fime and B for fhe ofher. 

Our dafa ineludes pafienfs who have been assigned fo medieafions ofher fhan mefformin or sul- 
fonylureas as an inifial fherapy. Figure 5 presenfs fhe sensifivify analysis for fhe esfimafion of fhe 
frealmenf effeel of inferesf based on a four dimensional sensifivify paramefer , ttu, ctis). 

The paramefer idenfifies fhe probabilify fhaf a pafienf wifh a parfieular value of BMl would 
have faken D = B ifZ = B (i.e., seeing a physieian wifh sulfonylureas preferenee) given fhaf she 
has faken D = A wifh Z = A (i.e., seeing a physieian wifh mefformin preferenee). The parame¬ 
fer aiB has a similar inferprefafion and fhe paramefer (ig"*) is defined as a ehanee of being 
a eomplier for pafienfs who have been assigned fo Z = A{B) and have faken D = A{B). Sinee 
fhe assoeiafion of sulfonylureas wifh weighl gain is known if makes sense fo assume fhaf fhe 
frue value of fhe paramefer aiA should be negative and aiB should be posifive. Also on average 
pafienfs who have faken sulfonylureas (D = B) given Z = B are more likely fo be eompliers eom- 
pare wifh fhose who have faken mefformin (D = A) given Z = A. This means fhaf if is very likely 
fhaf yf^ <yB^. 

The range of fhe frealmenf effeel eslimale varies by differenl ehoiees of yf^ and y^^. Speeifi- 
eally, for y^^ = 0.2 and y^^ = 0.6, fhe frealmenf effeel lies in [—0.5,1]; while for y^^ = 0.5 and 
yp = 0.3, if lies in [—0.7,0.5]. Based on our sensifivify analysis, if is unlikely fhaf metformin 
be assoeiated with inerease in BMl and a large negative effeet (i.e., E[T^ — TRIPS' = 54]) is also 
unlikely beeause it requires very small and large values of aiA and a\B, respeetively. Henee, if 
anything, the treatment effeet of interest should lie in [—0.6, —0.1]. 
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Fig. 5: THIN data: Sensitivity analysis for comparing the effect of metformin with sulfonylureas 
on BMI. Numbers on the plot present treatment effect estimates. 

6. Discussion 

This manuscript shows that IV analysis methods may fail to provide an unbiased estimate 
of treatment effects when analyzing data in which subjects are selected based on their received 
treatment. Specifically, this selection bias happens if the chance of including patients in the pre¬ 
selected data varies based on their IV value and/or some unmeasured confounders. For example, 
suppose we are interested in studying the comparative effect of treatment A and B while there is 
a third choice of treatment C. Suppose patients with more severe stages of the disease are more 
likely to be treated with C if seeing a doctor with A preference. Then if we analyzed a data that 
only includes patients who have received treatment A or B, severe cases will be under represented 
among patients who have seen doctors with A preference. This means that in the preselected data, 
the IV is associated with an unmeasured confounder which makes the IV invalid and results in a 
biased treatment effect estimate. 

Within the principal strata framework, we develop a sensitivity analysis that can be used to 
estimate the treatment effect among compliers as a function of a vector of sensitivity parameters. 
Specifically, fhe sensitivity parameters are used to identify the probability of being a complier 
given the IV value, received treatment and the observed outcome. The dimension of the sensi¬ 
tivity parameter can be reduced if it is plausible to assume that there is no always C takers. This 
is because the proportions in the principal strata are identifiable which means and can be 
estimated using the observed data. 
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Table 4: Simulation study: Sensitivity analysis of 0 = 0.80. The true veetor of sensitivity pa¬ 
rameters in seenario 1 and 2 are ,ai a, 0(,\b) = (y, 1,2), and (|, 1,2), respeetively. 


Seenario 1 Seenario 2 


Presumed am) 

Bias 

S.D. 

MSB 

Bias 

S.D. 

MSB 

(0.3,0.6,1,2) 

-0.37 

0.29 

0.22 

-1.92 

0.34 

3.79 

(0.3,0.6,0,2) 

1.37 

0.24 

1.95 

-0.18 

0.30 

0.12 

(0.3,0.6,1,0) 

-1.54 

0.27 

2.45 

-3.07 

0.34 

9.55 

(0.3,0.6,0,0) 

0.23 

0.24 

0.11 

-1.32 

0.27 

1.83 

(0.5,0.5,1,2) 

0.46 

0.26 

0.28 

-1.09 

0.29 

1.28 

(0.5,0.5,0,2) 

1.65 

0.26 

2.80 

0.11 

0.27 

0.08 

(0.5,0.5,1,0) 

-0.95 

0.27 

1.04 

-2.57 

0.28 

6.72 

(0.5,0.5,0,0) 

0.20 

0.23 

0.09 

-1.35 

0.26 

1.89 

(0.2,0.8,1,2) 

-1.22 

0.31 

1.59 

-2.76 

0.39 

7.80 

(0.2,0.8,0,2) 

0.87 

0.25 

0.81 

-0.69 

0.27 

0.55 

(0.2,0.8,1,0) 

-1.86 

0.29 

3.54 

-3.42 

0.36 

11.88 

(0.2,0.8,0,0) 

0.23 

0.23 

0.11 

-1.32 

0.25 

1.79 

(0.7,0.2,1,2) 

1.98 

0.34 

4.03 

0.44 

0.32 

0.30 

(0.7,0.2,0,2) 

2.72 

0.33 

7.54 

1.20 

0.32 

1.53 

(0.7,0.2,1,0) 

-0.55 

0.24 

0.36 

-2.09 

0.27 

4.45 

(0.7,0.2,0,0) 

0.18 

0.25 

0.10 

-1.34 

0.26 

1.88 


Appendix 

In this Appendix, we present the supplementary materials. 


Appendix: A 

Table 4 shows the bias (0.80 — 0), standard error and mean squared error of the treatment effect esti¬ 
mation for different combinations of the tme and presumed sensitivity parameters when the tme treatment 
effect 0 = 0.80. 
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